Skip to content

Conversation

@ggerganov
Copy link
Member

@ggerganov ggerganov commented Sep 24, 2025

Realized that the logic for reordering in the Metal backend is general enough to allow reorders even for nodes which dependencies might not be represented in the graph. The reason is that we keep track of the actual sources and destinations (either memory addresses or tensors) which is more general than the graph dependencies.

For example, the order of these ops would be preserved:

ggml_build_forward_expand(gf, ggml_cpy(a, view(b));
ggml_something(b or another_view(b));

Reordering of CPY ops is useful in Whisper graphs.

@github-actions github-actions bot added ggml changes relating to the ggml tensor library for machine learning Apple Metal https://en.wikipedia.org/wiki/Metal_(API) labels Sep 24, 2025
@ggerganov ggerganov merged commit 4ea0079 into master Sep 25, 2025
63 of 65 checks passed
@ggerganov ggerganov deleted the gg/metal-allow-reorder branch September 25, 2025 08:29
pwilkin pushed a commit to pwilkin/llama.cpp that referenced this pull request Sep 25, 2025
struct pushed a commit to struct/llama.cpp that referenced this pull request Sep 26, 2025
yael-works pushed a commit to yael-works/llama.cpp that referenced this pull request Oct 15, 2025
pwilkin pushed a commit to pwilkin/llama.cpp that referenced this pull request Oct 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Apple Metal https://en.wikipedia.org/wiki/Metal_(API) ggml changes relating to the ggml tensor library for machine learning

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants